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ABSTRACT 



A sequential process-pipeline (12) has a first processing 
stage (30) coupled to a CODEC (24) through a plurality 
of buffers, including an image data input buffer (28), an 
image data output buffer (26), and an address buffer 
(34). The address buffer stores addresses, each of which 
identifies an initial address of a block of addresses 
within an image memory (22). Each block of addresses 
in the image memory stores a block of decompressed 
image data. A local controller (IB) is responsive to the 
writing of an address into the address buffer to initiate 
the operation of the CODEC to execute a Discrete 
Cosine Transformation Process and a Discrete Cosine 
Transformation Quantization Process. 

24 Claims, 5 Drawing Sheets 
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twccn processing elements. No methodology is de- 

PROCESS-PIPEUNE ARCHITECTURE FOR scribed for optimizing a pipeline architecture so as to 

IMAGE/VIDEO PROCESSING efficiently operate with blocks of data. 

It is thus an object of this invention to provide a 

CROSS-REFERENCE TO RELATED PATENT 5 process-pipeline architecture wherein data buffering 

APPLICATION and local control functions are inserted within the proc- 

This patent application is related to commonly as- ess-pipeline at the boundaries of discrete process-pipe- 

aigned U.S. patent application Ser. No. 07/894,3 12, filed 1"« functions. 

Jun. 4, 1992 entitled "Block Control Method and Appa- It is a further object of this invention to provide a 

ratus for Digital Image CODEC Processors", by T. 10 process-pipeline architecture wherein discrete stages of 

Horvath et al. the pipeline include local data and possibly address 

~ ^ -«r^ t buffers, and a local controller to supervise local syn- 

FIELD OF THE INVENTION chromzation and timing. 

This invention relates generally to image processing It is one further object of this invention to provide a 

systems and methods and, in particular, to parallel pro- 15 system for encoding and decoding blocks of video/im- 

cosing systems that employ pipelined image processing age data, and to provide a methodology to optimize a 

techniques. pipeline architecture. 

BACKGROUND OF THE INVENTION SUMMARY OF THE INVENTION 

Constraints that are. encountered in Single Instruc- 20 The foregoing and other problems are overcome and 

tion-Multiple Datastream (SIMD) and Multiple Instruc- the objects of the invention are realized by an image 

tion-Multiple Datastream (MIMD) parallel processing processing system constructed with a process-pipeline 

systems include the following. architecture having a plurality of serially coupled com- 

In SIMD systems, a partitioning of an mput data pu tational routines or functions. Buffers, such as First 
segment makes it difficult to achieve proper processor 23 Qut buffers md a Sute Ma _ 
load balancing during parallel execution of the data chjnc (JUSM> m between adjacent, serially 
segments Also, an mcrementaldaU bandwidth require- lcd routincs or functions . This sequential prom- 
inent on the input data path becomes a design burden j lin£ processing architecture minimizes the synchro- 
when it is denied 1 to increase the number of parallel F lcx 5 fa a MIMD environment, whUe 
P™^»;»to SIMD *»^ v . „ ... 30 also increasing the flexibility of the image processing 
n MIMD systems complexit.es typ.caliy exist in . accommodating incremental changes in func- 
data-flow communication and synchronization amongst ; .. . ... , ^ *■ Jl 

the plurality of processors. J 0 "** Addmg and inserting a new function or rou- 

These problems become especially apparent when ?" c » wcomphihcd by determining a new processing 

blocks of image data arc being processed, such as when 35 boundary m the serial process-pipeline structure, and 

image data is being compressed (encoded) or decern- ncW funCtJOn ° r r0utmc at dctermmed 

pressed (decked). In that a source of image data, such boundary- 

as camera, or a sink for image data, such as a display . Buflcrs ' such " F . IFOs ' ^ inserted between process- 
monitor, operate in real time with fixed image data m « stages to provide the pipeline structure with an 

transfer rates, the image data processing system should 40 abaitv » ^ with a wide range of data-flow fluctua- 
be capable of keeping pace with the required source or tions. 

sink image data transfer rate. That is, the image data In accordance with a further aspect of this invention, 

processing system must be able to process the image 8 L 00 * 1 Statc Machine (LSM) control block is inserted 

data at a rate that prevents a loss of input image data, at boundary between two adjacent processes so as 

and/or that prevents the generation of undesirable vi- 45 to manage local synchronization tasks, thereby over- 

sual artifacts when the image data is displayed. coming the problem related to the increased overhead 
To obtain an optimal processing performance from a required by a conventional host or dedicated co-proces- 

process-pipeline architecture, the pipeline data-How sor. The LSM employs FIFO flags, in conjunction with 

should be maintained as full as possible. However, this an inter-process handshaking protocol, to determine a 

becomes difficult due to data-flow fluctuations that 50 correct sequencing of processes in the process-pipeline 

typically occur during the operation of such systems. architecture. 

Also, overall supervision of pipeline control can repre- That is, this invention provides a sequential process- 
sent a costly overhead factor to a host system or a dedi- pipeline architecture with embedded buffers and Local 
cated supervisory co-processor. State Machine control between processes. In combina- 
U.S. Pat. No. 5,046,080, issued Sep. 3, 1991, entitled 55 tion, the buffers and the LSM provide dynamic storage 
"Video CODEC Including Pipelined Processing Ele- buffers and also provide data-flow control of a process 
merits" (J. S. Lee et al.) describes a videophone system sequence. These features beneficially increase the flexi- 
that provides videophone service within a narrow band bility of the system and the extensibility of the system to 
digital network: A host CPU 20 is coupled through a various applications including, but not limited to, image 
bus 21 and a common memory 24 to an image bus 28. 60 processing applications. 

The CODEC 25 is comprised of a plurality of process- More particularly, an aspect of this invention pro- 

ing elements (PEs) connected as a pipeline. Each PE vides a sequential process-pipeline for use in a data 

includes a digital signal processor (DSP) and performs a processing system. The sequential process-pipeline in- 

part of the function of image coding. FIFO memory eludes a plurality of processing stages each of which 

modules 64 are comprised of a flag circuit and FIFO 65 executes a Process on data organized as data blocks, 

memory. The flag circuit controls data flow between The plurality of processing stages are serially coupled 

processing elements. The common memory is employed one to another for passing data blocks therebetween. A 

to store messages and performs synchronization be- first one of the processing stages is coupled, at a data 
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processing system resource interception boundary, to FIG. 3(a) shows an exemplary content of the Input 

the data processing system for inputting data from and FIFO of FIG. 2 for an image data decode operation; 

outputting data to the data processing system at a first FIG. 3(A) shows an exemplary content of the Input 

data rate. A last one of the processing stages is coupled FIFO of FIG. 2 for an image data encode operation; 

to a data source and/or a data sink operating at a second 5 FIG. 3(c) shows an exemplary content of the Address 

data rate that differs from the first data rate. FIFO of FIG. 2 for the image data decode operation of 

Each processing stage includes a data buffer inter- FIG. 3(c); 

posed between the processing stage and an adjacent FIG. 3(rf) shows an exemplary content of the Address 

processing stage. The first processing stage further in- FIFO of FIG. 2 for the image data encode operation of 

eludes an interface data buffer interposed between the 10 FlG Wf 

data processing system resource interception boundary FIG. 3(e) shows an exemplary content of the Output 

and the first processing stage. FIFO of FIG. 2 for the image data encode operation of 

It is shown that a SIZE of each of the data buffers F* 0 , ^ 

that are interposed between adjacent processing stages ? G * ■ » a d«afled cvcuit block diagram of a pres- 

is determined in accordance with the relationship: 15 enUy preferred embodiment of the image data coding 

and decoding system shown in FIG. 2; 

SIZE OF BUFFER /= (RP t ~ RPi+ OxPmax. FIG. 5 is a logic flow diagram that illustrates a JPEG 

DCT (Joint Photographic Experts Group, Discrete 

where, for a processing stage that executes a Process/, Cosine Transform) -based image data coding model; 

where i is Process sequence number associated with the 20 and 

processing stage in the process-pipeline, FIG. 6 is a block diagram that illustrates m greater 

Pmax is a maximum time required to completely detail correspondence of the major functional 

process a data block with the Process/, Wocks of 4 with process-pipeline architecture 

RP/ is a data rate produced by the Process/, and of FIG - *■ 

RP/+ 1 is a data rate from a next following Proces- 25 DETAILED DESCRIPTION OF THE 

s/+i. INVENTION 

The sequential process-pipeline further includes a ... - . T 

local controller associated with each of the processing . ™* blowing Detailed I^nption of the Invention 

stages for controlling the execution of the Process by m 18 organized as follows: Section (A) describes, in refer- 

the orocessine stase J 30 ence to FIG. 1, a sequential process-pipeline architec- 

foaTresenfly preferred embodiment of this invention is » ^^jT^ffE ™JZ 

. , ... ' y ■ , , . „. - scribes, in reference to FIGS. 2, 3, and 5, the overall 

processing stages executes a Discrete Cosine Transfer- fa refercnc( . piGS 4 ^ tf describes m f ^ 

mation Quantization Process, and one of the processing ^ fl ^ cmbodimcnt c f the Video/Image 

stages executes a Process to reduce an entropy of the processmB pipeline 

quantized Discrete Cosine Transformation coefficients. y B v ^ 

In the presently preferred embodiment, the first one ^ Section (A) 

of the processing stages is coupled to a second one of pj G x k 5implificd Wock diagram ihovnng a 

the processing stages through a plurality of buffers, of m ^ processing system 10 and, in particular, 

including an image data input buffer, an image data showing a mu iti. s tage (Stage/_i to Stage/+,) sequential 

output bufTer, and an address buffer. The address buffer proce ss.pipeline 12 that is constructed and operated in 
stores addresses, each of which identifies an initial ad- 45 accordancc with the invention. A host processor 14 is 

dress of a block of addresses within an image memory. coup ied, at a System Resource Interception Boundary 

Each block of addresses in the image memory stores a (SRIB) 14* to Stage/.! of the process-pipeline 12. 

block of decompressed image data. The local controller ia general, the sequential process-pipeline 12 can be 

is responsive to the writing of an address into the ad- considered to be a scries of processing units that are 
dress buffer to initiate the operation of the processing 50 concatenated in a predetermined sequence of processes, 

stages that execute the Discrete Cosine Transformation jh c processes being partitioned based on their routine or 

Process and the Discrete Cosine Transformation Quan- function. The processes are intended to function in 

tization Process. concert to achieve a desired computational end result. 

BRIEF DESCRIPTION OF THE DRAWINGS „ J^^^^SfS^Jm, 

The above set forth and other features of the inven- » Local State Machine 18, and a Process block 20. The 

tion are made more apparent in the ensuing Detailed process-pipeline architecture of the invention is charac- 

Description of the Invention when read in conjunction texized by the systolic nature of the Local State Ma- 

with the attached Drawings, wherein: chines 18, each having associated buffers 16a, 166 func- 

FIG. 1 is simplified circuit block diagram showing a 60 tioning as a data reservoir. Controls and data are passed 

portion of an image processing system and, in particu- between Stages via the Process blocks 20 and the buff- 

lar, showing a multi-stage sequential process-pipeline ers 16a and 16b, under control of the LSMs 18. 

that is constructed and operated in accordance with the The buffers 16a and \6b and the self-timed, LSM- 

invention; driven processing control embodied in the sequential 

FIG. 2 is a simplified circuit block diagram showing 65 process-pipeline 12 are well-suited to perform a series of 

an embodiment of an image data coding and decoding image processing operations and/or other operations on 

system that is constructed in accordance with the proc- digital image data. This technique provides automatic, 

ess-pipeline architecture of FIG. 1; flexible process-pipeline control on a per-block basis 
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throughout the process-pipeline, assuming that the pro- thus integrates and accommodates system 10 resource 

cess is self-triggered in the correct sequence. timing. 

Various aspects of the operation of the process-pipe- The Buffers 16 between processes function to logi- 

line 12 are now described in detail cally smooth out "density fluctuations" in the data- 

PROCESS SYNCHRONIZATION 5 flow - ^ *»WK**ry Buffers 16 also serve another pur- 
pose: to separate a data-dependent process from a data- 
Baaed on a given requirement in data-throughput that independent process. For many signal processing rou- 
the process-pipeline 12 is intended to fulfill, and tracing ^ the rclativc Ic ratcs at mc d^.f^ nodes are 
•long the sequential processes in the process-pipeline ^ mdcvcndent ^ ^pccl of the invention that 
id * tf£T *** PrOCCiSmg ° f " prOCeSS * 10 is embodied in the sequential process-pipeline 12 is the 
Themaximum required time, expressed in a number ^guston* of data-dependent processes from data- 
of processing cycles for each computational routine or ^epenaent processes, 
function, referred to herein as "Pmax", represents a decuio f ,n where ' 0 dcfine \P/°~" 
largest number of processing cycle counts required to » lnd how l ° determine the associated Buffer 16 sue 
completely process a block of data. The magnitude of uaoa « st 5ubsc ^ ucnt P roccsscs dlctated bv &«on 
Pmax may thus be considered as a Global synchronize ^ can be classified in three categories: 
tion interval for every computational process. The re- ] * thc svstem "bitrauon overhead (i.e., Asys) in- 
sulting Global synchronous clocking, performed by the curred from various resources; 
Host 14, signals the individual Processes 20 when to 20 processing nature of the relative sample rates of 
begin processing. This signalling is accomplished by the tk* data-flow within a process (i.e., is a given pro- 
Ready line that is coupled across the SR1B 14a to the cess data independent?); and 
LSM 18 of Stage i— 1. 3. the data bandwidth requirement or constraint on 
Underlying the Global synchronization performed by the input and on the output of each process 20 in 
the Host 14 is an inter-process data-flow communica- the process-pipeline 12. 

tion that is locally managed by each embedded Local In order to accommodate a short-term rate mismatch 
State Machine 18. The self-activated Local State Ma- between two adjacent processes 20, the size of the inter- 
chine 18 that regulates and synchronizes each of the posed Buffers are determined in accordance with the 
partitioned processes in the process-pipeline 12 executes 3Q expression: 
the protocols, described in detail below, in determining 

the correct timing to achieve process synchronization. SIZE of BUFFER/={iiP/-#iP/ + i)xPnux. (1) 

It should be noted that the number of processing 

cycles for a non-computational event, such as the pro- where 

cessing cycles required to perform a shared system 35 Pmax is the Processing time interval, or maximum 

memory resource arbitration, can at times dominate the processing time, for each computational process, 

total processing effort. The non-computational event is . RP/is a data rate produced by the Process/, 

thus preferably accommodated with an override mode. RP/+i is a data rate from the next following Proces- 

Herein, a maximum time, or number of cycles, to ac- % i+h and 

commodate the non-computational event is designated 40 j a a Process sequence number in the process-pipeline 

*'Tmax", which is a system 10 resource-dependent van- 12. 

able. Tmax can exist if and only if a system 10 resource For achieving continuous data-flow in the process- 

arbitration overhead (or maximum time required to pjpehM n ^ sizc of a Buffer 16 that nxcixcto the 

perform resource arbitration), referred to herein as SRIB 144 is considered separately. When intersection 

"Asys", is greater than Pmax. In FIG. 1, the Ready 45 ^ Aem 10 

resources occurs. Pmax in Equation (1) 

signal line 14b that is coupled between the Host 14 and is laccd b Tma*, and the SIZE-SRIB of the inter- 

the first LSM 18 of the sequential pipeline is employed f ^ im Buffer ft SRnj fa dctermined m accor _ 

as an Asys indicator, and serves as an override to the . . , . . . 

otherwise employed Pmax of the LSM 18. The Ready the " latlonshl P : 

signal line 14* thus provides, when enabled. Global » sbe-SPJB.« w -jv #+ i)xU««. C) 
lynchromzation of the process-pipeline 12. 

The protocols for each process can thus be described ^ accordancc ^ ^ e foregoing m aspect of the 

as follows m the exemplary pseudejodr invention provides a method of defining a^equential 

j^NOT EMPTV^*™ Buffer/- 1 pnemi ^ M for ^ m a daU prec^^tem. 

Tmax = Asys- The method includes the steps of (a) defining a plurality 

start the PROCESS/for every Tmax cycle; of daU processing stages each of which executes a Pro- 

ELSE IF Asys is not greater than Pmax AND IF the ° n d f u or 8 ani ^ cd » data blocks. Each of the stages 

Buffer,-! is NOT EMPTY a defined so as to, by example, segregate data-depend- 

start the PROCESS/ for every Pmax cycle; END 60 CTl Proccsscs from data-independent Processes. A next 

IF. step of the method (b) serially couples the plurality of 

The maximum system-dependent processing time (i.e. defined data processing stages one to another in an 

Tmax), by default, overrides and triggers the LSM 18, order selected for accomplishing a desired Processing 

which otherwise employs the local self-timed criteria function. A third step of the method provided, at an 

represented by Pmax. That is, a process must sometimes 65 interface between adjacent ones of the serially coupled 

wait upon slower system arbitration until the longest data processing stages, a date block Buffer and a local 

processing cycle (i.e. Tmax) enables the process to start. controller for controlling the transfer of data blocks 

This process-pipeline 12 mode of dynamic regulation across the interface. 
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having to consider a Global time schedule for the pro- 
Section w cesses propagating through the process-pipeline 12. 
FIG. 2 is a simplified circuit block diagram showing Encode Processinz 
an embodiment of an image data coding and decoding 

system that is constructed in accordance with the proc- 5 For encoding (compression), uncompressed source 
ess-pipeline architecture of FIG. 1. images are initially stored in the Image Memory 22. The 

The purpose of the system is to perform compression Processor 30 first prepares the CODEC 24 for encode 
(encoding) and decompression (decoding) of image processing by loading internal tables and performing 
data. For encoding, an image in an Image Memory 22 is other necessary functions. The encode process is con- 
passed to a Compression/Decompression (CODEC) 10 trolled by the LSM 18. Prior to initiating the encoding 
device 24, and the resulting compressed data is placed in of a block of data, the LSM 18 reads the Input FIFO 28 
an Output FIFO 26 for subsequent storage. When de- to retrieve the block header (H). The header, containing 
coding image data, compressed data in an Input FIFO CODEC 24 control information, is then stored in the 
28 is passed to the CODEC 24, and the resulting decern- Register 36 and applied to the CODEC 24. The 

pressed data is stored in the Image Memory 22 for dis- « Processor 30 then loads the Address FIFO 34 with 
play and/or subsequent processing. A local Processor ^ a f Memory 22 *>urce addresses for each image data 
30 is coupled between a local Memory 32 and the Out- b _J°^ pT , OC€SScd - , , . - w 

put FIFO 26, the Input FIFO 28, and an Address FIFO „™- **> * ows "° f^^S ° f ? C ^ 

34. The Address FIFO 34 stores the initial addresses of ™> » * r te "S* e cncodc f™}* 
blocks of decoded image data within the Image Mem- w «. *■* pO 28 contains a^pluraUty of Header 
ory 22, as described below. A Control RegisteV 36 has « ch f of which C f 24 ""f? 

an input coupled to the output of the Input FIFO 28 for jf>™tion for an associated data block of miage data 

receiving and storing data block Header (H) informa- that * ii0J * w ^f^Tf? ^fT^r 

tiao. This Header mformation includes CODEC 24 „ f 'T^^!?* ° f * C t ^rTor^ 

. f A . , , , , , 25 2 for the Input FIFO 28 contents of FIG. 3(b). FIG. 3(e) 

control information, also as described bdovv^ exemplary content of the Output FIFO of 

In this embodiment of the invention the CODEC 24 nG % ^ ^ ^ 

££5?? ^ " n " 8e d ^ dl S* te f SmC t T f ° mi After providing the Header Information for a block 
COmpr l SS1 ° n A presently pre- of ^ K data to ^ Register ^ ls M 18 ^ 

ferred DCT approach .s one phased on the Joint Photo- J0 a ^ ^ to ^ j M M tQ rcad a Wock 
graphic Experts Group (JPEG) baseline, as described in f . ^ t ^ CODEC 24. The CODEC 24 
ISO/IEC JTCI/SC2/WG8 CCITT SOVIII JPEG-8- "J^lto ^ data block and passes the com- 
R5.2 May 10, 1990. This algorithm divides a source ^ „, data to ^ 0utput mFO M for subse . 

image into 8x8 blocks of image data elements, and then quent rctricva i by thc Processor 30. When the CODEC 
processes each of the blocks. The processing of a block 35 u complex the compression of a full block of data, it 
of image data is shown in the symmetric logic How gcneratcs „, End of Block (EOB) signal to the LSM 18 
diagram of FIG. 5. i n response, the LSM 18 repeats the above described 

Prior to compression, image data is level shifted by operations by retrieving the Header information for the 
subtracting 128 (8 bit precision). A DCT device pro- ncxt data block from ^ FIFO 28, if there 
cesses blocks of 64 events and produces coefficients ^ are morc t0 ^ processed. The LSM 18 dcter- 

which are then quantized in accordance with a 64 entry mines tfo at there are more blocks to be processed by the 
lookup table of weights. The weights are based on a pre sencc of a further Image Memory 22 source address 
known response of the human eye to various frequen- u the Address FIFO 34, as indicated by the state of the 
cies. The result is a frequency dependent quantization Address FIFO 34 EMPTY signal line, 
which is then coded through a lossless entropy coding 45 

technique. Huffman variable length coding (VLC), as Decode Processing 

described by D. A. Huffman, "A Method for the Con- For decoding (decompression), compressed blocks of 
st ruction of Minimum-Redundancy Codes", Proc. IRE, image data are placed in the Input FIFO 28 by the 
September 1952, pp. 1098-1101, and run-length coding Processor 30. 

(RLC), as described by W. K. Pratt, Digital Image 50 , The Processor 30 initiates and maintains a record of 
Processing, John Wiley &. Sons, pp. 632, (1978), are two any required block control parameters, such as those 
most-often employed lossless entropy coding tech- referred to below in regard to the DCTQ 42 and DCT 
niques. Both of these techniques (VLC and RLC) take 40 control bits. The decode process is controlled by the 
advantage of the fact that the stream of DCT coeffici- LSM 18. Prior to initiating the decode of a block of 
ents frequently contains long runs of zeros. 55 data, the LSM 18 reads the Input FIFO 28 to retrieve 

The afored escribed characteristics of the process- the Header of the block of image data to be processed, 
pipeline 12, when used in a CO DEC -based image pro- The Header, containing CODEC 24 control informa- 
cessing system, provides the following advantages. tion. is then stored in the Control Register 36 and ap- 

Because of the inherent synchronization mechanisms, plied to the CODEC 24. The Processor 30 also loads 
the system 10 user need not be concerned with the exact 60 the Address FIFO 34 with Image Memory 22 target 
timing of the data transfers, so long as the processes in addresses for each image data block to be processed, 
the process-pipeline 12 occur in correct sequence. FIG. 3(a) shows an exemplary content of the Input 

Also, the FIFO memory inserted between processes FIFO 28 for an image data decode operation. As can be 
smoothes out fluctuations in the instantaneous transfer seen, the Header information is interspersed with blocks 
rates of the data flow. 65 of image data and End of Block (EOB) identifiers. FIG. 

Furthermore, the self-activated Local State Machine 3(c) shows an exemplary content of the Address FIFO 
Control makes between-bound ary processing wait 34 of FIG. 2 for the Input FIFO 28 contents of FIG. 
states transparent, and thus relieves a programmer from 3(a). 
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Following this operation, the LSM 18 again reads the 3. Huffman VLC to reduce the entropy of the quan- 
Input FIFO 28 to retrieve the first word of compressed tiled DCT coefficients. 

data for the block. The CODEC 24 begins decoding the The system illustrated in FIG. 4 partitions these tasks 
data and sends a READY indication to the LSM 18 for such that the DCT 40 performs the nonadaptive DCT 
additional data. The LSM 18 continues to read the 5 function, the DCTQ 42 performs the quantization func- 
Input FIFO 28 in response to each CODEC 24 tion, and Run Length Coding (RLC), and the Processor 
READY indication. The CODEC 24 identifies the 30 performs the VLC decompression function. As such, 
EOB for a block of image data read from the Input *&d referring briefly to FIG. 6, the Processor 30 corre- 
FIFO 28 and indicates this condition to the LSM 18 by sponds to the Process/ of FIG. 1, the DCTQ 42 corre- 
driving the EOB signal line. The LSM 18 waits until the 10 8 P° nds to the Process/ + l of FIG. 1, and the DCT 40 
decoded block of image data is transferred from the corresponds to the Process,-^ of FIG. 1. 
CODEC 24 to the Image Memory 22, and then repeats I" general, the VLC function that is executed by the 
the above-described process if there are more blocks to Processor 30, when encoding image data, is a table 
be processed, as indicated by the state of the Address lookup operation that corresponds to a mapping be- 
FIFO 34 EMPTY status signal line. 13 twecn source symbols and variable-length codewords. 

When decoding image data, VLC is generally accom- 
Section (C) plished by tracing along a coding tree until the decoded 

FIG. 4 is a detailed circuit block diagram of a pres- ***** ■* reached. The VLC coder maps the input data 
ently preferred embodiment of the image data coding mt0 v *ri*ble-lengtb codewords. As was noted above, 
and decoding system shown in FIG. 2. 20 for basclillc JPEG method that is employed in the 

This embodiment of the invention is provided on a presently preferred embodiment, the Huffman coder is 
plug-in circuit card intended to be interfaced with a ^ d to r f? ucc th 5 cnU ^ of *■* Qu^ed DCT 
data processing system having a system bus that oper- f,cients ' ™*} "» the HufIman coder reduces the average 
ates in accordance with the Micro Channel interface numb " of . b J te ! h * 1 ^present a symbol, without destroy- 
specification (Micro Channel is a registered Trademark 25 ° n « ul ? . 

of the International Business Machines Corporation of Fo ' mz& ? da J? ^^ata* compressed image data k 
Armonk NY) moved, under Host 14 control, from a Host bus 46, 

In this' embodiment, the Local Memory 32 and Buf- a ™* S g em , BuB 

fers,- (FIFOs 26 and 28) are sized in accordance with the m ^tA^J^ *l ^ M r*" 

description provided above in Section (A). More specif. 30 J™? ?™ 52 "f ^ stained v«thm the confines of 

ically, the circuit card contains three IK by 18 bit t\t f S couple together through a 

FIFOS 26, 28, and 34, and a 256 Kbyte Static Ram S t y " Cm Bu * GatC 54 SynchroiuzaUon of data transfers 

/cDAf\ ii_ i i vi tC • . V, , at the system resource interception boundary 14a. is 

(SRAM) as the Local Memory 32. The circuit card also ^ two flags ^ m cont ™Z within 

contains ,a loedttocessor 3£and ^two further process- 33 the Interface 48. Specifically, a Ready (RDY) flag 48a is 

SnS?^?^ 4 °'i >C I Q « 2) ^ "Ti thC «tbytheHostl4tomdi M tethatablockofcompressed 
CODEC 24. Devices 40 and « conform to the Joint ^ data ^ ^ stored m ^ SRAM 32< rdy 

Photograph* Experts Group (JPEG) baseline specifi- flag ^ is read by me Processor 30 and is used to initi- 

^J 00 * >i * j L . ate the decompression of the block of image data by the 

In a presently preferred embodiment of the invention ^ DCT 40 and the DCTQ 42 
the DCT device 40 is a L64730 Discrete Cosine Trans- At the tcnnmation of thc 0 f a block of 

form Processor that is avuhUe from LSI Logic of ^ data b ^ DCT 40, the DCTQ 42, and the VLC 
Mikpetas, Calif., and thc DCTQ device 42 is a L64740 performed by Processor 30, an Acknowledge (ACK) 
DCT Quantization Processor also available from LSI flag 48A is set by the Processor 30 to indicate that a 
Logic. The construction and operation of thc DCT 40 is 45 block 0 f compressed image data has been stored within 
described in a publication L64730 (July 1990), and the SRAM 32. The ACK flag 48A is read by the Host 14 

construction and operation of the DCTQ 42 is de- and & to the movC raent of the compressed 

scribed in a publication L67440 (September 1990). of ^ t0( by example, a mass storage 

These publications are also available from LSI Logic. device 15. 

?? T *f f ***** devices mclude Lntcmal dat * buffers 50 The local Processor 30 is embodied within a Digital 
(BUFF 40c and BUFF 42a). signal Processor (DSP) that performs interface func 

An external 64 byte translation buffer 44 operates in tions, Header parsing on the compressed data stream, 
conjunction with the DCT 40 and the DCTQ 42, as M d VLC. One suitable embodiment for thc DSP Pro 
described below. cessor 30 is a TMS320C30 device that is available from 

Control logic is contained within Programmable 55 Texas Instruments. It should be realized that a Digital 
Logic Devices (PLDs), specifically a SYSTEM CTRL Signal Processor is not required to implement the inven- 
LSM 18a and an XLATE CTRL LSM 186. The circuit tion, and that any digital data processor having the 
card, functions to perform image data encode/decode required speed and processing robustness may be em- 
specific processing. ployed. A Reduced Instruction Set (RISC) processor is 

The aboved escribed JPEG compression/decompres- 60 another suitable processor-type, 
sion algorithm that is illustrated in FIG. 5 can be di- DSP Processor 30 performs thc VLC decompression 
vided into three tasks. function and composes blocks of Run Length Coded 

1. The nonadaptive Discrete Cosine Transform (RLC) image data from the data stored within the 
(DCT) based on 8-by-8 blocks of image elements. SRAM 32. Processor 30 also produces a Header (H) for 

2. The uniform midstep quantization of the DCT 65 each block. The Header contains, on a per-block basis, 
coefficient using weighting functions that are opti- setup data for the DCT device 40 and the DCTQ device 
mized for human visual perception and compacted 42. The resulting data blocks are transferred by the DSP 
by Run Length Coding (RLC). Host 30 to the data input FIFO 28 as 1 6-bit words. Each 
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data block is preceded by its corresponding setup digital image data at 30 frames per second. The DSP 
Header word, and terminated by an EOB symbol, as Processor 30 prepares the circuit card for encode pro- 
shown in FIG. 3(a). Each block of a data may include cessing by loading the Input FIFO 28 with DCT and 
from one to 64 words of Run Length Coded (RLC) DCTQ setup Headers for each block (FIG. 3(6)), load- 
image data. 5 ing the DCTQ 42 with the appropriate quantization 
The DSP Processor 30 first initializes the DCTQ 42 tables, and then loading the Address FIFO 34 with 
by loading internal quantization tables and then, for Image Memory 22 target addresses for each block 
each block, provides an Image Memory 22 target ad- (FIG. Z(d)). 

dress for the block. The target address is loaded into the The SYSTEM CTRL lBo initiates the encode pro- 
address FIFO 34, as shown in FIG. 3(c). This address is 10 cess when it detects the presence of at least one address 
later used to store the decompressed block of image in the Address FIFO 34; The SYSTEM CTRL 18c 
data in the Image Memory 22. reads the first address from the Address FIFO 34 and 

The SYSTEM CTRL 18a initiates image data pro- transfers the address to the Address Latch of the Mem* 
cessing when it detects that at least one address is con- ory Control 22a Subsequently, the SYSTEM CTRL 
tained in the Address FIFO 34. This condition is indi, is 18a initiates the access of the corresponding block of 
cated by the EMPTY status flag of the address FIFO 34 image data in the Image Memory 22. The SYSTEM 
being deasserted when the Processor 30 loads the first CTRL 18a then reads the first block's Header from the 
block address. In response to EMPTY being deasserted, Input FIFO 28 and stores the Header in REG 36. The 
the SYSTEM CTRL 18a reads the first block's Header stored Header information is applied to the DCT 40 and 
from the Input FIFO 28 and stores the Header informa- 20 the DCTQ 42 to control encode processing parameters 
tion in the register (REG) 36. REG 36 outputs control for the image block, as described above . Image data is 
signals that are applied to the DCT device 40 and to the read, under control of the XL ATE CTRL 186, from the 
DCTQ device 42 to set their internal characteristics for Image Memory 22 into the buffer 44, which then passes 
processing the ensuing block data output from the Input 64 byte blocks to the DCT 40, also under control of the 
FIFO 28. 25 XLATE CTRL 186. In the encode mode, the buffer 44 

Specifically, the DCTQ 42 receives four control bits provides synchronization of the input data stream to 
from REG 36, the four bits being connected to the both the DCT 40 and DCTQ 42. Coefficients output by 
MODE.0, COMP0, COMP1, and RESPRED input the DCT 40 pass through the DCTQ 42 and become a 
terminals of the DCTQ 42. These terminals are used for stream of compressed 16-bh image "events". The image 
controlling the following functions, as described in the 30 "events" are stored in the Output FIFO 26 for subse- 
above mentioned DCTQ publication L67440. quent retrieval and VLC by the DSP Processor 30. The 

MODE.0 Selects one of two internal quantization DSP Processor 30 has an input coupled to the EMPTY 
tables flag of the Output FIFO 26 for detecting when the first 

COMPO, COMF1 Selects a DC predictor byte of compressed image data is stored therein, 

RESPRED Resets the internal DC predictors when 35 So long as an address is present in the Address FIFO 
high 34, the SYSTEM CTRL 18a continues to control of 

The DCT 40 receives one control bit from the REG reading of block addresses from the Address FIFO 34 
36, the control bit being connected to an INTER input and block Headers from the Input FIFO 28, as well as 
terminal. The signal on this terminal, when high, indi- the writing of encoded image block data to the Output 

cates that the device is to operate in an inter-frame 40 FIFO 26. 

mode. As can be seen, the SYSTEM CTRL 18a is triggered 

In operation, the DCTQ device 42 produces 64 coef- by the presence of the first address to initiate the local 
ficients per data block which, after passing through the processing of image data, and continues to control the 
DCT device 40, become 64 bytes of decoded image processing of image data so long as an image data block 
data. In that the DCT device 40 produces a scrambled 45 address exists within the Address FIFO 34. Thus, the 
(but consistent) ordering of the output data, a transla- use of the Address FIFO EMPTY status flag decouples 
tion is performed by the data buffer 44. This translation the operation of the fixed processing cycle Stages 
is accomplished by the XLATE CTRL 186 controlling (DCTQ 42 and DCT 40) from the variable processing 
the address inputs of the data buffer 44 so as to store the cycle Stage embodied in the Processor 30. Similarly, the 
decoded image data in a desired order. The reordered 50 use of the EMPTY status flag of the Output FIFO 26 
64 bytes of decoded image data are then stored within decouples the Processor 30 from the fixed cycle DCT 
the Image Memory 22, beginning at the address con- and DCTQ stages. Furthermore, in conjunction with 
tained in the Address FIFO 34. This address is applied the operation of the Processor 30, all image data pro- 
through a Gate (GT) 58 to the address bus (ADDR) cessing (coding and decoding) is performed nsynchro- 
portion of the Image Bus 52, and is latched in an Ad- 55 nously to, and in parallel with, other processing tasks 
dress Latch (AL) contained within a Memory Control- that may be performed by the Host 14. 
ler (MEM CTRL) 22a. The contents of the AL are It should be realized that although the description has 
automatically incremented so as to store the entire 64 focussed on the processing of a single block of data, that 
byte decoded data block. A Dynamic RAM Controller during normal operation a number of blocks are typi- 
(DRAM CTRL) 226 controls the timing of the access- 60 cally moving through the process-pipeline. For exam- 
ing of the 4 Mbytes of Dynamic RAM that comprise the pie, compressed image data is stored within the SRAM 
Image Memory 22. 32 awaiting decompression, the Processor 30 is per- 

This process continues until the Address FIFO 34 forming the VLC operation on a data block being pre- 
becomes EMPTY, indicating that there are no further pared for decompression, one or more data blocks al- 
blocks of compressed image data to process. 65 ready VLC'd by the Processor 30 are enqueued within 

For image data encoding, source images are con- the Input FIFO 28, and the CODEC 24 is processing a 
tained in the Image Memory 22. The source images may current data block under control of the Header informa- 
be obtained from, by example, a camera that generates tion that is associated with the current data block. 
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Referring to FIG. 6, there is shown a correspondence 
between the presently preferred embodiment of FIG. 5 
and the novel process-pipeline architecture of FIG. 1. 
At can be seen, the synchronization of the data flow at 
the system resource interception boundary 14a is ac- 5 
coroplished by the Processor 30, in conjunction with 
the RDY and ACK flags 48a and 486, respectively. The 
SRAM 32 functions as the input and output buffers 
0—1) of FIG. 1. Sta ge/is comprised of the Processor 30, 
the SYSTEM CTRL LSM 18c. and the Input and Out- 10 
put FIFOs 28 and 26, respectively. The processing that 
occurs within Stage /(VLC) is data dependent, and re- 
quires a variable number of processing cycles. Synchro- 
nization of the data flow between Stage/and Stages 1 is 
accomplished by the SYSTEM CTRL LSM 18a, in 15 
conjunction with the EMPTY flag of the Address 
FIFO 34 (not shown in FIG. €). Stage/+i and Stage, +2 
are comprised of the DCT Quant 42 and the DCT 40, 
respectively. These stages operate in a data-independent 
manner and have a fixed number of processing cycles 20 
for each data block. Buffers, >i of FIG. 1 are imple- 
mented with the internal buffers 40a and 42a of the 
DCTQ 42 and the DCT 40. Control of the data move- 
ment to and from the 64 Byte Buffer 44 and the Image 
Memory 22 is accomplished by the XLATE CTRL 25 
LSM 18d. 

Although described above in the context of a system 
for compressing and decompressing blocks of image 
data in real time, the process-pipeline architecture of 

FIG. 1 may be also be applied to data processing appli- 30 source interception boundary (SRIB) is determined in 



last one of said processing stages being coupled to 
a data source and/or a data sink operating at a 
second data rate that differs from the first data rate; 

each of said processing stages including data buffer 
means interposed between the processing stage and 
an adjacent processing stage, said first processing 
stage further including interface data buffer means 
interposed between the data processing system 
resource interception boundary and the first pro- 
cessing stage; and wherein 

a SIZE of each of said data buffer means that are 
interposed between adjacent processing stages is 
determined in accordance with the relationship: 

SIZE OF BUFFER/-(JU»f-*P( + l)X/Wx, 



where, for a processing stage that executes a Pro- 
cess,-, where i is Process sequence number associ- 
ated with the processing stage in the process-pipe- 
line, 

Pmax is a maximum time required to completely 

process a data block with the Process/, 
RP/is a data rate produced by the Process* and 
RP/+1 is a data rate from a next following Proces- 

V+h 

2. A sequential process-pipeline as set forth in claim 1 
wherein, for the first processing stage, a SIZE 13 SRIB 
of said interface data buffer means at the system re- 



cations other than image processing. Also, a number of 
modifications may be made to the presently preferred 
embodiment of the invention that is shown in FIG. 5. 
For example, the operation of the image processing 
system may be enhanced with the addition of a block 64 33 
that performs motion compensation in accordance with 
the MPEG (Motion Picture Experts Group) standard. 
For this enhancement, two frames of image data that 
are stored within the Image Memory 22 are subtracted 



accordance with the relationship: 

SIZE_SfUB = Rp, + ,)x Tmax. 

where, 

Tmax is a maximum time required to accomplish a 

non-computational event. 
3. A sequential process-pipeline as set forth in claim 1 
and further including local control means associated 



and are processed by the DCT 40. Also, other functions 40 with each of said processing stages for controlling the 



may be provided by coupling a video Interface 62 to the 
image bus 52, and exporting and importing image data 
through the video interface 62. These other functions 
may include, and are not limited to, image scaling, color 
conversion, filtering, rotation, recomposition, and 43 
depth cueing. Similarly, the DCT 40 and the DCTQ 42 
may be replaced by other image processing circuitry to 
perform functions other than image data encoding and 
decoding. 



execution of the Process by the processing stage. 

4. A sequential process-pipeline as set forth in claim 1 
wherein said data blocks are comprised of image data. 

5. A sequential process-pipeline as set forth in claim 4 
wherein one of said processing stages executes a Dis- 
crete Cosine Transformation Process, one of said pro- 
cessing stages executes a Discrete Cosine Transforma- 
tion Quantization Process, and wherein one of said 
processing stages executes a Process to reduce an en- 



Thus, while the invention has been particularly 50 tropy of quantized Discrete Cosine Transformation 



shown and described with respect to preferred embodi 
menu thereof, it will be understood by those skilled in 
the art that changes in form and details may be made 
therein without departing from the scope and spirit of 
the invention. 53 

Having thus described our invention, what we claim 
as new, and desire to secure by Letters Patent is: 

1. A sequential process-pipeline for use in a data pro- 
cessing system, comprising: 
a plurality of processing stages each of which exe- 60 
cutes a Process on data organized as data blocks, 
said plurality of processing stages being serially 
coupled one to another for passing data blocks 
therebetween, a first one of said processing stages 
being coupled, at a data processing system resource . 65 
interception boundary, to the data processing sys- 
tem for inputting data from and for outputting data 
to the data processing system at a first data rate, a 



coefficients. 
6. An image processing system, comprising: 
a sequential process-pipeline comprised of a plurality 
of processing stages each of which includes means 
for executing a Process on image data organized as 
image data blocks, said plurality of processing 
stages being serially coupled one to another for 
passing image data blocks therebetween, a first one 
of said processing stages being coupled to a data 
processing system for inputting compressed image 
data from and for outputting compressed image 
data to the data processing system; wherein 
said first one of said processing stages is coupled to a 
second one of said processing stages through a 
plurality of buffer means, including an image data 
input buffer means, an image data output buffer 
means, and an address buffer means, said address 
buffer means storing addresses each of which iden- 
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tiftcs an initial address of a block of addresses 
within an image memory means, each block of 
addresses storing a block of decompressed image 
data. 

7. An image processing system as set forth in claim 6 5 
wherein said second one of said processing stages is 
further coupled to a third processing stage, said second 
and third processing stages each including means for 
performing a decompression Process of a block of 
image data; wherein said processing means of said first 10 
processing stage stores at least one block of compressed 
image data within said input buffer means and an ad- 
dress within said address buffer means, the address iden- 
tifying the initial block of addresses within said image 
memory means wherein the decompressed image data is 13 
to be stored at the completion of the decompression 
Process. 

8. An image processing system as set forth in claim 7 
and further comprising means, responsive to the storage 
of the address within said address buffer means, for i 
initiating said processing means of said second and said 
third processing stages to perform the decompression 
Process on the block of image data stored within said 
input buffer means, ^ 

9. An image processing system as set forth in claim 8 
wherein said processing means of said first processing 
stage also stores information in said input buffer means 
for specifying decompression Process control informa- 
tion to said processing means of said second and said 3Q 
third processing stages, and wherein said initiating 
means further includes means for reading out the infor- 
mation and for applying the information to said process- 
ing means of said second and said third processing 
stages. 35 

10. An image processing system as set forth in claim 
6 wherein said second one of said processing stages is 
further coupled to a third processing stage, said second 
and third processing stages each including means for 
performing a compression Process on a block of image 40 
data; wherein said processing means of said first pro- 
cessing stage stores information in said input buffer 
means for specifying compression Process control infor- 
mation to said processing means of said second and said 
third processing stages, said processing means of said 45 
first stage further storing an address within said address 
buffer means, the address identifying an initial address 
of a block of addresses, within said image memory 
means, wherein is stored a decompressed block of 
image data that is to be compressed. 30 

11. An image processing system as set forth in claim 
10 and further comprising means, responsive to the 
storage of the address within said address buffer means, 
for initiating said processing means of said second and 
said third processing stages to perform the compression 55 
Process on the block of image data stored within said 
image buffer means. 

12. An image processing system as set forth in claim 
10 wherein said processing means of said second pro- 
cessing stage stores the compressed block of image data 60 
into said output buffer means, and wherein said process- 
ing means of said first processing stage is responsive to 
the storage of compressed image data within said output 
buffer means to read out said compressed image data. 

13. An image processing system as set forth in claim 65 
6 wherein said second one of said processing stages is 
further coupled to a third processing stage, and 
wherein: 



said third processing stage executes a Discrete Cosine 
Transformation Process on a block of image data to 
generate a plurality of coefficients; 

said second- processing stage executes a Discrete Co- 
sine Transformation Quantization Process on a 
block of image data; and wherein said first process- 
ing stage executes a Process to reduce an entropy 
of the quantized Discrete Cosine Transformation 
coefficients. 

14. An image processing system, comprising: 
means for interfacing said image processing system to 

a Host data processing system, said interface means 
including a memory buffer means for storing com- 
pressed image data that is received from, and that is 
output to, the Host data processing system; 

first processor means coupled to said memory buffer 
means for composing into blocks of compressed 
image data the compressed image data that is re- 
ceived from the Host data processing system; 

image buffer means for storing uncompressed image 
data; and 

means for compressing and decompressing (CO- 
DEC) blocks of image data, said CODEC means 
being interposed between said image buffer means 
and said first processor means, said CODEC means 
being coupled to said first processor means through 
processor buffer means, said processor buffer 
means including an image data input buffer means, 
an image data output buffer means, and an address 
buffer means, said address buffer means storing 
addresses each of which identifies an initial address 
of a block of addresses within said image memory 
means, each block of addresses storing a block of 
decompressed image data. 

15. An image processing system as set forth in claim 

14 wherein said first processor means stores at least one 
block of compressed image data within said input buffer 
means and an address within said address buffer means, 
the address identifying the initial block of addresses 
within said image memory means wherein the decom- 
pressed image data is to be stored at the completion of 
the operation of said CODEC means in decompressing 
the at least one block of compressed image data. 

16. An image processing system as set forth in claim 

15 and further comprising means, responsive to the 
storage of the address within said address buffer means, 
for initiating said CODEC means to decompress the 
block of image data stored within said input buffer 
means. 

17. An image processing system as set forth in claim 
15 wherein said first processor means also stores infor- 
mation in said input buffer means for specifying, for the 
block of image data, decompression control information 
to said CODEC means. 

18. An image processing system as set forth in claim 
17 and further comprising means, responsive to the 
storage of the address within said address buffer means, 
for initiating said CODEC means to decompress the 
block of image data stored within said input buffer 
means, and wherein said initiating means further in- 
cludes means for reading out the information and for 
applying the information to said CODEC means prior 
to the operation of said CODEC means in decompress- 
ing the block of compressed image data. 

19. An image processing system as set forth in claim 
14 wherein said first processor means stores information 
in said input buffer means for specifying compression 
control information to said CODEC means and also 
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stores an address within said address buffer means, the 
address identifying an initial address of a block of ad- 
dresses within said image memory means, the block of 
addresses storing an uncompressed block of image data 
that is to be compressed. 

20. An image processing system as set forth in claim 
19 and further comprising means, responsive to the 
storage of the address within said address buffer means, 
for initiating said CODEC means to compress the un- 
compressed block of image data stored within said 
image buffer means. 

21. An image processing system as set forth in claim 
14 wherein said CODEC means includes: 

a second processor means for executing a Discrete jj 
Cosine Transformation Process on a block of 
image data to generate a plurality of coefficients; 

a third processor means for executing a Discrete 
Cosine Transformation Quantization Process on a 
block of image data; and wherein 20 

said first processor means executes a Process to re- 
duce an entropy of the quantizied Discrete Cosine 
Transformation coefficients. 

22. A method of defining a sequential process-pipe- 
line for use in a data processing system, comprising the 23 
steps of: 

defining a plurality of data processing stages each of 
which executes a Process on data organized as data 
blocks, each data block being comprised of a plu- 
rality of data elements that are operated oh as a 
group, each of said stages being defined so as to 
segregate data-dependent Processes from data- 
independent Processes; 

serially coupling the plurality of defined data process- 
ing stages one to another in an order selected for 
accomplishing a desired Processing function; and 

providing, at an interface between adjacent ones of 
the serially coupled data processing stages, data 
block Buffer means and a local control means for 40 
controlling the transfer of data blocks across the 
interface. 
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23. A method of defining a sequential process-pipe- 
line for use in a data processing system, comprising the 
steps of: 

defining a plurality of data processing stages each of 
which executes a Process on data organized as data 
blocks, each of said stages being defined so as to 
segregate data-dependent Processes from data- 
independent Processes; 

serially coupling the plurality of defined data process- 
ing stages one to another in an order selected for 
accomplishing a desired Processing function; 

providing, at an interface between adjacent ones of 
the serially coupled data processing stages, data 
block Buffer means and a local control means for 
controlling the transfer of data blocks across the 
interface; 

coupling one of the data processing stages, at a data 
processing system resource interception boundary, 
to the data processing system for inputting data 
from and for outputting data to the data processing 
system at a first data rate; and 

coupling a second one of the data processing stages to 
a data source and/or a data sink operating at a 
second data rate that differs from the first data rate. 

24. A method as set forth in claim 23 and further 
including the step of operating a stage/ of the sequential 
process-pipeline in accordance with the criteria: 

IF Asys is greater than Pmax AND IF a Buffer/- 1 is 
NOT EMPTY 
Tmax=Asys; 

start a PROCESS/ for every Tmax cycle; 
ELSE IF Asys is not greater than Pmax AND IF the 
Buffer/- lis NOT EMPTY 
stan the PROCESS, for every Pmax cycle; END 
IF; wherein 

Asys is a maximum time required to perform a data 
processing system resource arbitration, 

Tmax is a maximum time required to accommodate a 
non-computational data processing event, and 

Pmax is a maximum time required for processing the 
PROCESS/. 
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